I really like this. I didn't think we could use nix because portable /usr/bin/env doesn't support extra arguments, but you've proven me wrong 🎉.

My only objection as far as the design is concerned, is that this isn't about shells at all, but on closer reading, this isn't specific to the dubious nix shell command at all. In fact, this seems to solve #801.

To support nix develop, you could override the various basePath/baseDir variables to point to the script's location. That way its functioning does not depend on the caller's working directory, which could be anywhere.

Another issue that has come up as I'm writing the docs (stealing most of it from nix-shell), is that we don't run the setup or shell hooks to modify PYTHONPATH or PERL5LIB and so forth. Not sure if we want to adopt exactly the legacy nix-shell behavior which led us down an convoluted path. Is there an easier way?

we don't run the setup or shell hooks

This is why we need to rename the current nix shell to something like nix env run.
If you want to run stdenv, currently your only nix-command option is nix develop.

I would like Nix to defer to packages, allowing it to take "installing to environment variables" seriously. Maybe it's overkill because for regular commands PATH+XDG_DATA_DIRS will generally do, but perhaps this idea can be used for more complicated stdenv-like environment composition. Assuming Nix can somehow learn how to compose variables. (Not everything is :-separated)

Previously my (and apparently Eelco's) assumption was that we don't need the stdenv/setup-based ad-hoc shells like you are asking for. What's your use case? For packaging we've assumed writing an expression is best.

Anyway that's all fairly orthogonal to the shebang implementation, as long as we don't

automatically inject "shell" argument

I hope I was able to provide some context.

   #!/usr/bin/env nix
   #! nix shell --quiet
   #! nix nixpkgs#bash
   #! nix nixpkgs#shellcheck

I find utterly confusing references to derivations are written as #! nix f#pkg instead of #! nix shell f#pkg or simply #! f#pkg.

I think we should also support a varadic derivation references until the end of the line (e.g. #! nix shell f#pkg1 f#pkg2) like nix shell already does.

The #! nix has the same role as #!nix-shell which seemed to have served a technical purpose for potentially mixing shebang aware interpreters. I wonder to what extend this is required. Perhaps it could be optional? For example, we could strip either #! or #!nix, whichever is a longer prefix. That way, we only need to confuse the reader when it's necessary for technical reasons.

Note that #! nix with the space would then be parsed as a #! prefix and a nix argument instead of #! nix as a prefix. I think this is an improvement, because #!nix reads a bit more like a single keyword.

Example:

#!/usr/bin/env nix
#! shell --quiet
#! nixpkgs#bash
#! nixpkgs#shellcheck

And in case we're invoking an interpreter that can and must ignore #!nix lines, but not #! lines.

#!/usr/bin/env nix
#!nix shell --quiet
#!nix nixpkgs#bash
#!nix nixpkgs#shellcheck
#! coolLang foo bar baz use imagination

The proposed approach is variadic and can be put all on one line. Sometimes these lines end up very long and having them split up can be helpful.

As to the prefix, this was meant to copy the way nix-shell does it( most of the code was simply copied or adopted over).

There are several shebang-aware interpreters, eg perl. I’m sure there are more.

The biggest issue is being unable to load the stdenv in the “right way”. I suspect that is actually what people will want. The examples of legacy nix-shell show bringing in python and Perl libraries and that will only work if we run shellHooks or at least do more than just modify PATH.

Forgive me if I am wrong, but is it really necessary considering env -S option:

-S, --split-string=S
              process and split S into separate arguments; used to pass multiple arguments on shebang lines

That way your example becomes:

#!/usr/bin/env -S nix shell --quiet nixpkgs#bash nixpkgs#shellcheck nixpkgs#hello --ignore-environment --command bash
# shellcheck shell=bash
set -eu
shellcheck "$0" || exit 1
function main {
    hello
    echo 0:"$0" 1:"$1" 2:"$2"
}
"$@"

Aside from multi-line support, I don't see any problem here.

@ymeister that's a good suggestion, but it is not universally supported on the other unixes. It also doesn't allow spaces in arguments for use with --expr.

Forgive me if I am wrong, but is it really necessary considering env -S option

The original nix-shell shebang mechanism did not rely on that feature - i presume for various compatibility reasons. If one wishes to use it for their own scripts, they are welcome to do so. It should work with this PR (if not, I'd consider that a bug).

My greater concern is that examples such as (https://github.com/NixOS/nix/blob/7ab3bc32b139e251ff8d9abf25d6cadf3c0c6945/doc/manual/src/command-ref/nix-shell.md#use-as-a--interpreter) won't work with nix shell as it is currently implemented. Thus this is involved with the whole nix-shell/nix shell/nix develop/nix run/nix env do-a-thing design and debate. My summary of the situation is that there are a few use cases:

1. Please put some packages A, B, C into my PATH. - usually nix-shell -p somepkg now nix shell
2. I want to develop on package A. - usually nix-shell -A somepkg now nix develop
3. Please make language-specific libraries available: eg: PYTHONPATH, PERL5LIB... - usually nix-shell -p or nix-shell shell.nix
4. Just run a program, don't change any env vars - nix run

To make (1)(3) possible nix-shell would create a derivation behind the scenes on your behalf, composing it together and additionally invoking stdenv environment setup and shell hooks. This is why nix shell nixpkgs#python3 nixpkgs#python3Packages.requests does not work as expected. The shebang example from this PR hits this issue. The only way to express "please run the shellHooks" is to create your own flake devShell, but ad-hoc CLI mechanisms don't seem to allow it.

Another approach

• Can we add "multiple installables" support to nix develop? Then nix develop is always the one that brings in stdenv/shellHooks.

This would create three distinct tools:

• nix run - only runs an executable, no changes to env vars, multiple installables makes no sense
• nix shell - only adds things to PATH, multiple installables means to add them all to the PATH
• nix develop - add things to PATH and run shell hooks. multiple installables means to do a "buildEnv" composition.

@roberth that's a good notice. Albeit the wide adoption of env -S among other unixes, it doesn't seem to be POSIX-compliant, so that's a bummer. On another note, I haven't experienced any problem using it with --expr:

#! /usr/bin/env -S nix shell --impure --expr "with import <nixpkgs> {}; python.withPackages (pkgs: with pkgs; [ prettytable ])" -c python

import prettytable

# Print a simple table.
t = prettytable.PrettyTable(["N", "N^2"])
for n in range(1, 10): t.add_row([n, n * n])
print t

(with import <nixpkgs> {} requires nixpkgs flake path in $NIX_PATH. That can be achieved either by pinning it system-wide (NixOS), or by exporting NIX_PATH="nixpkgs=$(nix flake metadata nixpkgs --json | jq -r .path)")

Above code also showcases an example of a workaround of the issue discussed by @tomberek. Although I agree that it is just a workaround and is nowhere near a proper solution like one proposed by @tomberek.

I may have been wrong about -S with quotes. I only read the man page, not the info page (:confused:) which goes into more detail, confirming your observations for at least GNU env and FreeBSD.
Instead of <nixpkgs> you should be able to reference a local flake when the base path is passed correctly. Also note that nix shell --expr is very different from nix-shell --expr (without and with stdenv respectively; also use case (1) vs (2))

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-shell-with-shebang/17088/2

Please make sure these 2 keep working, Thanks.

#!/usr/bin/env nix-shell
//! ```cargo
//! [dependencies]
//! time = "0.1.25"
//! ```
/*
#!nix-shell -i rust-script -p rustc -p rust-script -p cargo
*/
fn main() {
    for argument in std::env::args().skip(1) {
        println!("{}", argument);
    };
    println!("{}", std::env::var("HOME").expect(""));
    println!("{}", time::now().rfc822z());
}
// vim: ft=rust

#!/usr/bin/env nix-shell
#![allow()] /*
#!nix-shell -i bash -p rustc
rsfile="$(readlink -f $0)"
binfile="/tmp/$(basename "$rsfile").bin"
rustc "$rsfile" -o "$binfile" --edition=2021 && exec "$binfile" $@ || exit $?
*/
fn main() {
    for argument in std::env::args().skip(1) {
        println!("{}", argument);
    };
    println!("{}", std::env::var("HOME").expect(""));
}
// vim: ft=rust

For anyone stumbling onto this in the future, you can avoid having to set NIX_PATH by using the following (as roberth hinted):

#!/usr/bin/env -S nix shell --impure --expr "(import (builtins.getFlake \"nixpkgs\") {}).python3.withPackages (ps: [ ps.requests ])" --command python

Reviewed in Nix team meeting 2023-05-01:

• status unclear
• @roberth: stuck on syntax?
• @edolstra: not a replacement for nix-shell -p yet
• @roberth: that's ok, separate scope
• @roberth: can technically be done with nix develop --expr '...'
• @tomberek: what about the syntax? Currently scans all comment-like lines
• @tomberek will rebase, @roberth still assigned for review

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-05-01-nix-team-meeting-minutes-51/27803/1

Oops, I didn't reset the upstream pushRemote. Made a separate PR with my additions. I'll revert this in a sec.
EDIT: github doesn't seem to tell me the old hash, or at least I'm not sure. I'll leave it at these rebased commits.

I've changed the quoting and interpretation of relative paths in #8327. wdyt?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flox-nix-community-update-2023-04/28233/1

I've changed the quoting and interpretation of relative paths in #8327. wdyt?

Looks reasonable, but more contentious. I'm okay with either approach.

Would it be desirable to have support for brace expansion? So you can do something like this:

#!/usr/bin/env nix
#!nix shell --command bash
#!nix nixpkgs#{bash,jq,fd}

v2 was merged, please try it out and provide feedback